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This report is an assessment of the overall impact and influence on education of 
137 terminated Planning and Operational Grants made under the ESEA Title III 
Projects to Advance Creativity in Education (PACE) program. Analysis and evaluation 
seek to determine whether (1) individual project objectives were identified and 
achieved, (2) PACE participation resulted in educational changes or improvements, (3) 
school districts were motivated to continue their programs through local funding and 
initiative, and (4) expenditures, efforts, and activity generated by PACE are justified in 
terms of outcomes. To accomplish these objectives, an evaluation instrument 
(appended) is used to investigate project characteristics, project accomplishments, 
provisions for continuation, project design, and final appraisals. Major problems 
mentioned most often in conjunction with the projects include project finance, public 
relations, lack of clear planning and objectives, lack of qualified personnel, problems 
of implementation, and lack of official cooperation. The overall appraisa’ rated 76 
projects as outstanding or good while only 61 were rated as average or below. Nine 
specific recommendations are made to aid the quality and continuation of future 
PACE projects. (TT) 
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Introduction 



Terminal reporting has become a matter for concern now that 
many projects are coming up to the three-year cutoff point. It is in- 
teresting to note that the first USOE guidelines (1965) that were given 
in the manual for project applicants did not even make mention of 
terminal considerations, and the 1966 and the 1967 revised versions 
do mention an end-of-grant report but give only the barest suggestions 
as to what it should contain. 

This report is an effort to determine the’ 'contents of 137 
randomly selected terminated planning and operational ESEA Title 
III grants. 

One should take into account the fact that most of the 137 pro- 
jects represented early-funded projects. Later projects, beginning 
with the second year, have displayed a more sophisticated approach. 

Also, our paper analys'.s should be considered in light of the 
great difficulty of judging field success — where it really counts of 
the project as opposed to the paper picture. Members of the PACE 
national study team have been impressed by what they found in the 
field as compared with the project proposals. So one must view find- 
ings from analyses of terminal reports, no matter how carefully 
undertaken, with some skepticism. 
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With an understanding of the limitations of our "paper tiger," 
nevertheless, we believe it represents a useful dimension of this 
second national effort. Every project is required to submit a termi- 
nal report. From these efforts, one should be able to glean some- 
thing about how public money has been used to improve public educa- 
tion. 

A four-man study steam was coordinated by Charles F. Mar- 
tin, assistant professor of education. University of Kentucky, who also 
did a major portion of the final analysis. The team members brought 
varied and well-rounded backgrounds to the study, with two members 
having a total of 32 years of public school teaching and administration, 
and the other two having predominately college teaching and research 
experience. The other three team members were: Russell Bowen, 
graduate assistant. Department of Curriculum and Instruction, Univer- 
sity of Kentucky; Art Lucky, psychologist. State Department of Health, 
Frankfort, Kentucky; and John R. Payne, assistant professor of edu- 
cation, "^^irginia Polytechnic Institute. 

This study is sponsored by an ESEA Title III grant, to the Cen- 
ter for Effecting Educational Change, Fairfax County, Virginia, which 
subcontracted the assignment to the University of Kentucky's Research 
Foundation. 

Richard I. Miller 
Director of Study 
November 15, 1968 
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ANALYSIS AND EVALUATION 



OF 

137 ESEA TITLE III PLANNING AND OPERATIONAL GRANTS 

The general objectives of the study were to determine the over- 
all influence and impact on education of a sample of terminated ESEA 
Title III Operational and Planning Grants. The five major areas in the 
evaluation instrument included: project characteristics, project ac- 

complishments, provisions for continuation, project design, and final 
appraisals. (A copy of the evaluation instrument is included as Appen- 
dix A. ) Specifically, the study sought to: 

1. Provide a statistical report and summary of a sample 
of terminated Title III Operational and Planning Grants. 

2. Determine whether individual project objectives were 
identified and achieved. 

3. Determine whether PACE participation resulted in edu- 
cational changes and improvements in school districts, 
including any evidences of direct influences upon stu- 
dents. 

4. Determine v;hether school districts were motivated to 
continue their programs through local funding and 
initiative. 

5. Determine as much as a paper analysis will allow 
whether. the expenditures, efforts, and intensive 
activity generated by PACE are justified in terms of 
outcomes . 
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Planning and orientation sessions were held to acquaint team 
members with objectives of the study. 

1. The instrument developed for analysis of the reports 
covered five major areas and several related categories. In addition, 
the terminated planning grants were analyzed through use of an adden- 
dum covering the three specific areas of needs assessment, program 
development, and possibilities of converting the planning grant into an 
operational grant. The instruments provided 111 individual checklist 
opportunities for responses, plus 49 items in the addendum that were 
iipplicable only to planning grants. 

2. The sample consisted of 94 terminated planning grants 
and 43 terminated operational grants. Thirty-five states were repre- 
sented in the planning grants and 28 in the operational grants. The 
■“•rales included all of the geographical areas of the n< tion. 

3. The analysis consisted, in the first instance, of identify- 
ing terminated planning and operational grants and separating them 
from the other types of grants. This task was unexpectedly time con- 
r>-umijig and somewhat difficult due t(^the failure of the many reports 
to properly and fully complete the official application and report 
forms. The individual planning and operational grants then were 
given a code number and the analysis began. 

Analysis was accomplished by application of the guidelines in- 
strument. Following the completion of the 94 planning and 43 
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operational grants, various statistical sunanaaries were compiled. 
These will be presented in the following section of the report. During 
the actual analysis the team members conferred freely concerning 
problems of identification, meaning, intent, content, and so forth. 

This report will consist of the following six sections: statisti- 

cal summaries, project accomplishments, evaluation procedures, 
major problems, overall appraisal, and recommendations. 



STATISTICAL SUMMARIES 



This section includes a number of items that are included to 



assist the reader in understanding the sample of 137 projects upon 
which this study is based. 

These items are: 

Number of projects in each submission period 
A financial summary 

School enrollment and staff involvement 
Type of project 
Focus of activity 

Identification of projects by states 

Scope of project 

Area served 

Types of activity 

Types of project 

Provisions for continuation 

Number of Projects in Eac h Submission Period 



For planning grants 

The study covers three funding periods — fiscal year 1966, 1967, 
and part of 1968. From the first period in 1966 (projects numbering 
1-733), 11 projects are analyzed and processed; from the second peri- 
od of fiscal 1966 (projects numbering 734-1723), 47 projects are 
analyzed; and the third period (projects numbering 1724-2726), in- 
cludes 26 projects. Thus a total of 84 of the 94 planning projects 
analyzed come from the 1966 fiscal year funding period. 

For fiscal year 1967, eight projects are analyzed. Three are 
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from the first 1967 funding period (projects numbering 2727-3152), and 
five are from the second 1967 funding period (projects numbering 3153- 
4487). The sample includes two projects from the first period of fis- 
cal year 1968, encompassing projects numbering 4485-5795. 

The large number of planning grants in the 1966 fiscal year 
(91. 6 percent as compared to only 9. 4 percent for both 1967 and 1968) 
reflects the fact that ESEA was ir:'>.plemented in 1965 and became opera- 
tional in 1966. The impetus "to get in on a good thing, " the relative 
ease with which early planning grants were approved and funded, and 
the pressures from the public and educational establishment for school 
participation were factors that prompted submission of several hundred 
project proposals during the first funding period, or very shortly after 

f 

ESEA became a law. 

F or operational grants 

The first period of 1966 included 10 terminated operational pro- 
jects; the second period of fiscal 1966 was represented by 17 projects; 
and the third period of 1966 included 13 projects. Thus a total of 40 
projects of the 43 evaluated by the study team originated in fiscal 1966 
or immediately following enactment of the ESEA in 1965. The sample 
included one project from the first period of fiscal 1967, and two from 
the second submission period of 1967. 

The following table summarizes the data for both planning and 
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operational grants with respect to the number of projects in each sub^- 



mission period. 



Fiscal 1966 


Planning 


Operational 


Total 


First period 


11 


10 


21 


Second period 


47 


17 


64 


Third period 


26 


13 


39 


Fiscal 1967 


First period 


3 


1 


4 


Second period 


5 


2 


7 


Fiscal 1968 


First period 


_2 


_0 


2 


Totals 


94 


43 


137 



One might ask about the validity of this sample in view of the 
significant improvement in the quality of proposals submitted, during 
the second year. The question is a legitimate one, aud the study team 
was aware of it in making their analyses and recommendations, 

A Financial Summary 

For planning grants 

The 94 Title III planning grant projects received a total of 
$5, 295, 006 during the funding periods encompassed in this report. Of 
this amount, $588, 982 was for non-federal support, largely state or 
local funds; and the federal .support other than ESEA Title III totaled 
$53, 192. Actual federal support for Title III for the three year period 
included in this study totaled $4, 649, 747. 
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For operational grants 

The total fiscal allocation for the 43 ternninated operational 
grants was $4, 330, 579. Of this amount, $661, 335 came from non- 
federal sources. The actual Title III allocation totg,led $3,401,807, 
while federal support from sources other than Title III totaled 
$265, 947. Due to arithmetical errors in the financial section of 
some individual projects, the total in those reports do not balance 
exactly. 



The following table summarizes the data for both planning and 



operational grants with respect to project support. 



Type of 
project 



ESEA 


Other 


Non- 


Total 


Title III 


federal 


federal 





94 planning 
43 operational 
Totals 



4', 649, 747 
3,401,807 
8, 051, 554 



53, 192 
265, 947 
319, 139 



588, 982 
661, 335 
1, 250, 317 



5, 295, 006 
4. 330, 579 
9, 625, 585 



School Enrollment and Staff Involvement 
For planning grants 

The 94 planning projects reviewed by the study team encom- 
passed a geographic area containing an estimated total school popula- 
tion of 3, 845, 018 students. (See table on page 1 1. ) At the pre-kinder- 
garten level, 14,383 public and 1, 197 non-public school students were 
located in the geographic area served by this sample. The total num- 
ber of kindergarten pupils served in the geographic area included 



- 10 - 



156, 755 public school pupils and 11, 875 non-public school pupils. In 
grades one through six, 1, 532, 506 public school pupils and 275, 611 
non-public school pupils; and for grades seven through twelve, 

655, 724 public and 88, 517 non-public school pupils were totaled. 

The distribution of participants, according to race, is: 

2, 154, 760 white; 123, 947 negro; 11, 723 American Indian; and 61, 997 
other non- white — for a total of 2, 352, 43 1 . (See table on page 13.) 

F or operational grants 

The 43 operational projects encompassed a geographic area 
containing an estimated school enrollment of 1,494, 192 students. (See 
table on page 11.) At the pre-kindergarten level, an estimated 1,404 
public and 390 non-public school students were in the geographic area 
included in this sample. The estimated total number of kindergarten 
pupils in the geographic area included 26, 303 public school pupils and 
1, 663 non-public school pupils. In grades one through six, 745, 027 
public school pupils and 109, 884 non-public school pupils; and for 
grades seven through twelve the total was 524, 695 public school and 
42, 050 non-public school pupils. 

Again, the estimated number of pupils actually served by the 
projects is much smaller than the total geographic population. (See 
page 11.) At the pre- kindergarten level, 1,260 public and 184 non- 
public school pupils were served. At the kindergarten level, 11, 195 
public and 274 non-public school pupils were listed; for grades one 



Total Potential School Population in Geographic Area 
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through six, 226, 361 public and 14, 817 non-public school students were 
listed; and for grades seven through twelve the total was 156, 731 
public and 4, 192 non- public school pupils. 

The distribution of participants, according to race, is: 338, 123 

white; 47,420 negro; 3, 367 American Indian; 1, 789 other non-white — 
for a total of 410, 483. (See page 13. ) 

The preceding tables summarize data on school enrollment and 
staff involvement. 

The reader should not confuse these "ball park" figures with 
what might actually represent the scope of the project. 

The figures do represent some very rough indication of the 
maximum number of pupils and students that might be influenced by 
the project under the most favorable conditions. But this quantitative 
dimension must not be confused with quality, per se . 

Type of Project 

Of the 94 planning grants, 53 projects were designated as in- 
novative, 32 as exemplary, and two as adaptive. (These categories 
follow those set up by the USOE guidelines.) A total of seven proposals 
submitted failed to designate or identify the type of project. 

Of the 43 operational grants, 26 were designated as innovative, 
14 as exemplary, and three as adaptive. 
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The following table summarizes the data on this item: 



Type 


94 Planning 


43 Operational 


Total 


Innovative 


53 


26 


79 


Exemplary 


32 


14 


46 


Adaptive 


2 


3 


5 


No description 


_7 




7 


Totals 


94 


43 


137 


These categories 


remain elusive. 


The first national s 


tudy de- 


fined an educational innovation as "a new 


or different concept. 


methodol 



ogy, organization, or program that is systematically introduced into 
the classroom, school system and/or the State as a whole." 

In this sense many projects appeared to be an innovation, but 
many others seemed to be adaptive. Few could be judged as exem- 
plary. 



Focus of Activity 

F or planning pr oj ects 

A large majority of the projects — 83 in number — concentrated 
on program planning, and 18 projects conducted pilot studies as as- 
sociated activities. (Again, these categories were taken from USOE 
guidelines.) Eight projects planned construction, while seven were 
designated as operational, and one for remodeling. One project made 
no response in this category. (The number of activities listed exceeds 
the total number of projects because some projects sponsored more 



than one major activity.) 
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For operational projects 

Twenty-five operational projects were judged to have planning 
as a major activity or program focus. This finding is a little unusual 
in that one would expect most program planning to take place during 



the planning grant phase of the program. Thirty- seven projects con- 
centrated or program operation, while 10 conducted various types of 



pilot activities. Four were devoted primarily to construction of facili 
ties, three to remodeling, and two to planning of construction. (The 



number of activities exceeds the total number of projects because 



several projects focused on more than one activity.) 



The following table summarizes the findings*. 
Activity Planning Operational 



Planning of program 
Planning of construction 
Conducting pilot programs 
Operation of program 
Constructing 
R emodeling 
Not identifiable 
Totals 



83 

8 

18 

7 

0 

1 

1 

118 



25 

2 

10 

37 

4 

3 



81 



Total 



’108 

10 

28 

44 

4 

4 

1 

199 



Identification of Projects by State s 
For planning grants 

The 94 projects were located in 35 of the 50 states. The eight 
northeast states provided 27 of the projects used in the study, the mid 
west consisting of eight states provided 21 proposals; the southern 
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region consisting of nine states provided 19 projects; and the western 
region consisting of nine states provided 19 projects. 

California led in number of projects with nine; Pennsylvania 
followed with eight, and Michigan, Illinois and New Jersey each had 
five projects. Three states had four projects in the sample, seven 
states had three projects each, eight states had two projects each, and 
13 states had only one project. 

For operational grants 

The 43 projects were located in 28 of the 50 states. The west- 
ern area accounted for 12 projects; the south had 12 projects; the 
east had 10 projects; and the noirth had nine. 

California led in number of projects with six. Eight states had 
two projects, and 19 others had a single project. 

The following table- summarizes the distribution by states: 
Distribution of Planning and Operational 



Grants by States 



States 


Planning 


Operational 


Total 


Alabama 


1 


0 


1 


Alaska 


1 


0 


1 


Arizona 


0 


0 


0 


Arkansas 


1 


1 


2 


California 


9 


6 


15 


Colorado 


1 


0 


1 


Connecticut 


2 


1 


3 


Delaware 


2 


0 


2 


District of Columbia 


0 


0 


0 


Florida 


3 


2 


5 


Georgia 


4 


0 


4 


Hawaii 


0 


0 


0 
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States 


Planning 


Operational 


Total 


Idaho 


1 


1 


2 


Illinois 


5 


0 


5 


Indiana 


1 


0 


1 


Iowa 


0 


1 


1 


Kansas 


2 


2 


4 


Kentucky 


1 


2 


3 


Louisiana 


3 


0 


3 


Maine 


0 


0 


0 


Maryland 


1 


1 


1 


Massachusetts 


1 


2 


3 


Michigan . , 


5 


1 


6 


Minnesota 


3 


1 


4 


Mississippi 


0 


0 


0 


Missouri 


3 


0 


3 


Montana 


1 


0 


1 


Nebraska 


0 


0 


0 


Nevada 


0 


0 


0 


New Hampshire 


3 


1 


4 


New Jersey 


5 


1 


6 


New Mexico 


2 


1 


3 


New York 


3 


2 


5 


North Carolina 


2 


1 


3 


North Dakota 


0 


1 


1 


Ohio 


4 


2 


6 


Oklahoma 


0 


1 


1 


Oregon 


0 


1 


1 


Pennsylvania 


8 


2 


10 


Rhode Island 


4 


0 


4 


South Carolina 


0 


1 


1 


South Dakota 


0 


0 


0 


Tennessee 


2 


0 


2 


T exas 


3 


2 


5 


Utah 


1 


1 


2 


"V ermo nt 


0 


1 


1 


V irginia 


2 


1 


3 


Washington 


1 


0 


1 


West Virginia 


0 


0 


0 


Wisconsin 


1 


1 


2 


Wyoming 


2 


0 


2 


Unidentified 


1 


2 


3 
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Scope of Proj ect 

For planning grants 

The scope of the projects indicated that a majority (40) covered 
one district. Another 40 proposals concentrated on multi-district 
areas, with 26 in multi-district of seven or more, and 14 were in 
multi-distri jts of six or less. Only one regional (projects covering 
more than one state) and one state project were included in the sample. 
Five of the projects sampled involved a few schools in one district 
while four concentrated only on one school in a district. Three pro- 
jects failed to indicate the scope or limitations of their projects by not 
responding to this item. 

For operational grants 

None of the terminated operational projects were national in 
scope nor did any of them cover an entire region or state. Eleven of 
the projects focused upon one district, 14 functioned in multi-districts 
containing two to six different school systems, and six grants operated 
in multi-districts of seven or more. Seven projects were located in a 
few schools within a single district, and two projects operated each in 
one school. The scope of the terminated operational projects compares 
favorably with the distribution of the Planning Grants in that the majo- 
rity were located in either one district or fn multi-districts. Three 
projects failed to designate the scope of their projects, and were not 
included in the report. 



o 
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The distribution of planning and operational grants is given in 



the following table: 
Scope 



Planning 



Operational 



National 0 0 

Regional 1 0 

One state 1 0 

One district 40 11 

Multi-district (2 - 6) 14 14 

Multi-district (7 +) 26 6 

A few schools in one district 5 7 

One school 4 2 

Not identifiable 3 3 

Totals 94 43 



Total 



0 

1 

1 

51 

28 

32 

12 

6 

6 

137 



Area Served 



For planning projects 

Approximately 72 projects out of the 94 focused on urban areas 
while only 21 served ,'ural areas. This is not surprising since the 
needs of inner-city and large urban area schools are a major thrust 
for PACE. Twenty-three of the projects served central city areas and 
12 served urban fringe areas, and 20 projects were operative in cities 
of 10, 000 or more, while 15 served cities of 2, 500 to 10, 000. 

Twelve projects focused on rural areas-— communities of 1, 000 
to 2, 500 and only nine served areas designated as "other rural. " Three 
projects failed to designate the area served by failing to respond to this 




particular category. 
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For operational projects 

A majority of the projects (18) served either the inner-city or 
fringe areas; however, an almost equal number were located in subur- 
ban areas with 12 projects serving cities of 10, 000 or more and five 
serving cities of from 2, 500 to 10, 000 population. Rural sections had 
a much smaller percentage of projects with six located in rural areas 
of from 1, 000 - 2, 500 population, and only two operating in rural areas 
smaller than 2, 500 population. As in the case of the planning projects, 
a large majority of operational projects were concentrated in the in- 
ner-city and urban areas. 

The following table summarizes the areas served by the 137 
projects: 



Area 



Planning Operational 



Total 



Urban 

Central 
Urban fringe 



23 

12 



10 

8 



33 

20 



Other urban (suburban) 

Cities of 10, 000 and over 
Cities of 2, 500 to 10, 000 



20 

15 



12 

5 



32 

20 



Rural 

1, 000 to 2, 500 
Other rural 



12 

9 

94 



6 

9 

__0 

43 



20 

11 

3 



Unidentified 



Totals 



139 
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Types of Activity 

For planning grants 

The types of activity engaged in by the various projects were 
varied. Not surprisingly, activities related to planning, surveying 
needs, designing new programs and visiting innovations were most 
frequent, with 36 projects giving this category as their prime activity. 
The second most prevalent activity consisted of services to local 
schools with 21 projects, and the third most frequently listed cate- 
gory — with 18 projects — was instruction or services to pupils in the 
arts, social studies, guidance and counseling, and various other 
activities related to student instruction. Ten projects concentrated 
on services to teachers and principals, mostly in the areas of in-ser- 
vice training, audio- visuals and instructional materials, and seven 
projects emphasized the installation of one or two innovations in a 
single school or two, while two projects failed to indicate their pri- 
mary focus. 

F or operationa l grants 

A large majority of the 43 projects, however, concentrated upon 
instructional services to pupils in the arts, sciences, social studies, 
etc. , by providing mobile la,bs, ETV, visits to museums and guidance 
and counseling services. Six of the projects provided administrative 
planning, dissemination and developmental services to local school 



districts 
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The emphasis on various phases of instructional services is to 
be expected. Seven of the projects provided in-service programs to 
administrators and teachers in audio-visual training, provisions for 
instructional materials, and demonstrations of programs. Only three 
programs out of a V j projects surveyed involved the installa- 

tion of innovations in one or two schools, and no programs were rnn 
cerned with activities such as surveying needs, visiting other innova- 
tions, or planning for these activities. 

The following table summarizes the types of activities found in 
the 137 projects. (The total number of operationa.l projects exceeds 
the 137 figure because a few projects were judged as having two or 

more major functions.) 

Type of Activity 

a. Mostly instruction or ser- 
vices to pupils, such as 
arts, sciences, social stu- 
dies, mobile demonstra- 
tion, museum visits, ETV, 
outdoor camping, and 
guidance and counseling. 

bo Mostly pla.nning, such as 
surveying needs, design- 
ing new programs, and 
visiting innovations 

c. Mostly services to several 
local school districts, such 
as administration, dissemi- 
nation, planning, and de- 
veloping 



Planning Operational Total 



18 36 54 



36 0 36 



21 6 27 



o 

ERIC 
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Type of Activity Planning Operational Total 

d. Mostly services to teachers 

and principals — services 
such as instructional mate- 
rials, audio-visual, demon- 
strations, and in-service 
training 10 

e. Mostly installation of one or 

two innovations in one or two 
schools — innovations such as 
ITA, computer-assisted instruc- 
tion, teaching aides, parent in- 
volvement, job placement, and 
new courses 7 

f. Not indicated 2 

Totals 94 



Types of Project 

For planning proj ects 

The 94 PACE planning projects were almost evenly divided in 
terms of focus: Forty- six concentrated upon a sirgle idea or program, 
while 45 were planning to serve as supplementary centers usually with 
several programs or activities. Three projects failed to indicate an 
area of concentration. 

F or operational projects 

Twenty- seven of these projects focused upon a single idea or 
program, or about 62% of the total number. The remaining number 
of grants (16) concentrated on organizing or providing various types 
of supplementary educational centers. 



17 



52 



10 

2 



146 
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The following table summarizes the types of projects found in 
the 137 projects: 



Focus 


Planning 


Operational 


Total 


Upon single idea or program 


46 


27 


73 


Supplementary center program 


45 


16 


61 


No indication 




jO 


3 


Totals 


94 


43 


137 



Addendum for Planning Grants 

As was noted on page four of this report, a special addendum 
covering the specific areas of (1) needs assessment, (2) program 
development and (3) possibilities for converting plannLig grants into 
continuation grants, was used in analysis of the 94 planning grants 
sampled. 

The study team detei mined where a needs assessessment study 
had been made as an aspect of the planning grant. Thirty- six projects 
were judged to have done so, and 57 projects had not made a needs 
assessment study. 

A second part of this criterion determined what community and 
school factors were used in those 36 projects that did conduct an as- 
sessment study to determine their needs. The two leading community 
factors were (1) rural-urban composition and (2) socio-economic 
class, with 12 responses each. Eleven projects indicated that 






employment patterns were important community considerations, while 
nine projects listed attitudinal considerations as the most prominent 
community factor used. 

Ar '.ong the other community variables used by the study team 
were median family income, median grade attainment, and horizontal 
mobility, with four each; and the school variable, per pupil expendi- 
ture with five responses. Some six projects indicated that pupil and 
comm.unity needs were important community factors; however, this 
appears to be a dual category including both community and school 
factors . 

Turning to the school factors used in the assessment studies, 

14 projects used composition of the student body, student achievement 
measures and the socio-economic class structure of the school system 
were used by eight projects each. Five projects used study needs as a 
school factor in needs assessment. 

Community and school variables used by the 36 (out of 94) pro- 
jects that conducted needs assessment studies are summarized in the 



following table: 
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‘'s'* / 

Community factors— 



F r equency used 



a. Rural-urban composition 

b. Socio-economic composition 

c. Employment patterns 

d. Attitudinal considerations 

e. Per pupil expenditure 

f. Median family income 

g. Median grade attained 

h. Horizontal mobility 

i. Other 



12 

12 

11 

9 

5 

4 

4 

4 

3 



School factors Frequency used 

a. Composition of student body 14 

b. Socio-economic class structure 

of school system 8 

c. Student achievement measures 8 

d. Other ^ 



i 

I 



11 

The number of community factors used exceeds the total 
number of projects because several projects used more than one 
procedure to assess needs. 



PROJECT ACCOMPLISHMENTS 



The second category appraised by the study team covered 11 
items related to project accomplishments. 

Accomplishments on the 137 projects were rated in terms of 
what the project authors said was achieved. 

The 11 items were: 

Research 

Product development 

Skill development 

Process development 

Program development 

Pupil relations 

Community relations 

In-service programs for teachers 

Demonstration 

Dissemination 

Implementation 

Provisions for con' inuation 



Research 

Twenty-three terminal planning project reports, or 24 percent, 
had some type of research design that was related logically to the pro- 
gram. Fifteen of these 23 projects had accomplishments that appeared 
significant in terms of potential contributions. 

In three cases the accomplishments were considered to be over- 
stated, and in 1 1 projects the accomplishments werfe defined as 
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*’ordinary. " The accomplishments of two out of the 23 projects were 
rated as "trite or insignificant. " 

For 76 percent of the terminal reports, however, no research 
design or research attention was evident. 

Nineteen terminated operational project reports, or 44 percent, 
were judged as having research-oriented programs, or as having used 
research methodology in planning, operational procedures and/or 
evaluation. Research procedures in 18 of the 19 projects appeared to 
flow orderly and logically from the program, but in only one project 
did the claimed accomplishments in terms of research appear to offer 
significant contributions. 



Product Development 

This area refers to the development of new programs and ma- 
terials such as curriculum innovations or aids for the handicapped. 

Activities in 45 planning projects fitted into this category, and 
22 of this number were judged to have accomplishments flowing log- 
ically from the programs. Fifteen of the 45 were credited with having 
accomplishments that were significant in terms of potential contribu- 
tions. Accomplishments in three of the 45 programs were judged as 
being over- stated, and the accomplishments in one program were 
judged to be trite or insignificant. 

Product development activities in 16 operational projects 



seemed to originate and to flow logically from the project's overall 
program, and three projects in product development presented ac- 
complishments that seemed capable of making significant contribu- 
tions. One claim made by the project authors was judged to be "'trite 
or insignificant'’’’ by the reviewers. 

Skill Development 

Activities in eight terminal planning reports could be classified 
as skill development, which included activities such as new mathema- 
tics, foreign languages, and the Initial Teaching Alphabet (ITA) . Five 
of these eight were rated as flowing logically from the program, and 
three of the activities presented accomplishments judged to be signi- 
ficant. 

Fourteen operational projects had skill development in subject 
matter areas such as reading, mathematics, and the social sciences 
as their major goal. Of this number, 13 projects were credited with 
flowing logically from the program. One other project was judged to 
have made claims that were "trite or insignificant" and not likely to 

•V. 

make a significant contribution. 

Process Development 

Eleven projects stressed process development such as inquiry 
training and critical thinking. Five of the 11 projects were judged to 
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have accomplishments that could make potential contributions in pro- 
cess development; five were appraised as claiming accomplishments 
that were "ordinary;" and only one project was judged to have made 
an orderly or "flow" transition from the project. Four out of the 43 
operational projects had some form of process development as an inte~ 
gral part of their total efforts. These four programs were judged to 
flow logically from the project itself; however, none was judged as 
likely to make significant contributions to education in their area. 

Program Development 

Some 52 planning projects featured various types of program 
developments such as computer assisted instruction, non-graded 
schools, special science programs, and team teaching. One program 
accomplishment was judged to be "overstated;" ten were appraised as 
being "ordinary;" 18 programs seemed to flow logically from the pro- 
ject; and 22 were judged significant in terms of potential contributions. 

Four projects included some aspect of computer technology as 
a major dimension. One focused on computer assisted instruction as 
a supplement to the regular teaching methodology, and other two pro- 
jects utilized computer assistance in pupil accounting, guidance ser- 
vices and personnel recording. The small number of projects involving 
computer technology confirms the fallacious nature of some early esti- 
mates that said the bulk of PACE resources was going into hardware. 
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Seven projects used various media as integral parts of the pro- 
ject design and operation. Xwo were instructional television (ITV) pro- 
jects; one used rural and local artifacts in a museum setting as a part 
of ITV teaching, and the other used ITV to present various cultural en- 
richment programs. Video tapes and other materiaxs were used in 
one project, and two projects constructed models to demonstrate new 
programs to students and teachers. Another project planned to develop 
a system for classification, location and retrieval of specific instruc- 
tional materials. 

For the operational projects, program development in such 
areas as team teaching and organization of non-graded schools, etc. , 
was evident in 23 out of the 43 projects. Twenty-one of the 23 were 
appraised as being consistent with the proposal, and two were rated 
as offering potentially significant contributions to their field. 



Pupil R elations 

Twenty programs stressed pupil relations, referring primarily 
to programs that directly involved pupils. Ten of these programs ap- 
peared to flow logically from the project, and five projects were judged 
significant in terms of potential contributions. Five of the 20 were ap- 



praised as "ordinary. 



I I 



Operational programs featuring pupil relations were scarce, as 




was the case with this category for planning reports. For the 






- 33 - 



operationa.l grants, eight out of 43 projects were directly designed to 
further the relations between the school and the pupils outside the regu 
lar curriculum. Two of the eight were judged to flov^ directly and log- 
ically from the program, but six were appraised as making claims 
that were "trite or insignificant. " 

Community R elation s 

A relatively large number of planning projects — 67^ — claimed 
accomplishments in community relations. Three claims for accom- 
plishments were judged as "overstated, " 13 were classified as "ordi- 
nary, " and one was considered "insignificant." Twenty-eight projects 
listed accomplishments in community relations that appeared to flow 
logically from the stated program, and 22 were appraised as making 
significant contributions in terms of community relations. Many of 
the claimed accomplishments, however, are judged as superficial and 
were primarily "paper" involvements rather than meaningful commu- 
nity participation. Only four projects were judged as including in a 
significant manner the community, local teachers, and the pupils. 

Thirty-five operational projects claimed they had achieved ac- 
ceptable community relations in the areas where their projects func- 
tioned; however, the review team concluded that the claims made by 
the projects were only "trite or insignificant" in 23 out of the 35 cases 
and did not possess a great deal of innovative ingenuity. Nine others 
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were evaluated as being only "ordinary," while only three were cre- 



dited with having promising contributions toward better community 



relations 



In- s er vie e Programs for T eacher s 
Sixty- six terminal planning project reports included in-service 
training activities for teachers, administrators, and/or staff mem- 
bers. Accomplishments in 16 of the 66 projects were appraised as 
being "ordinary;" three were "overstated;" 25 programs seemed to 
flow logically from the projects; and 22 achieved significant results 
in terms of possible transfer. 

Four projects concentrated on a variety of in' service training 
programs for teachers and other professional school personnel. These 
projects included retraining of teachers by outside "experts" as well 
as instruction by co-workers who had taken special courses. In sever- 
al instances, teachers were included in the planning stages through 
conferences and discussion groups. In a few cases special emphasis 
was given to training teachers prior to major curriculum changes. 

The frequent inclusion of in-service education in planning pro- 
jects as well as the appearance of four projects that were concentrating 
on this area is encouraging. Too few projects, however, extensively 
involved local teachers in strategies related to planning, dissemina- 
tion, and implementation of an innovation. 




- 35 - 



The appraisal team found that 26 operational projects apparently- 
provided for some type of in-service training. Of this number, 11 pro- 
grams were rated as flowing naturally from the program structure, 
and six were judged capable of making significant contributions to in- 
cervice education. The claims of nine projects, however, were dis- 
credited and their accomplishments were judged "trite or ordinary. " 

As judged from the reports, not a great deal in the way of in-service 
education was accomplished. 

Demonstration 

Twenty- three planning projects featured provisions for demon- 
strating programs, such as team teaching and others. Of this number, 
eight claims for accomplishment were appraised as "ordinary," seven 
flowed logically from the program, and eight were appraised as offer- 
ing significant contributions. 

Nine operational projects described programs demonstrating 
innovations, such as team teaching. Of this number, the reviewers 
concluded that four flowed logically from the program or were an inte- 
gral part of the projects, four appeared capable of making significant 
contributions, and one was judged to be "trite or insignificant. " 

Dissemination 

Inall, 153 accomplishments indissemination were claimed, which 



o 
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indicates that some projects used more than one method of dissemina- 
tion. 

Accomplishments of three projects out of 94 were judged as 
"overstated," 55 of the claims were classified as "ordinary," and 
eight were judged to be "trite or insignficant. " Fifty-four were judged 
to flow logically from the project program, and 27 were ra.ted signifi- 
cant in terms of potential contributions. One program’s claim of ac- 
complishment was judged to come from a source that did not seem re- 
lated to the program. 

All 43 terminated operational projects said that efforts were 
made to disseminate the information about their project. While a 
rather wide assortment of dissemination strategies were employed, in 
only 12 situations did the procedures seem to flow logically from the 
program, and only nine projects presented accomplishments that ap- 
peared likely to produce significant contributions. Nineteen projects 
laid claims to dissemination that were judged to be "ordinary, " and 
claimed accomplishments of three projects were judged as "trite or 
insignificant. " The review team discredited the effectiveness of al- 
most 50 percent of the claimed dissemination efforts, and they believed 
that less than 20 percent of the claimed accomplishments were signifi- 



cant or potentially promising. 
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Implementation 

Sixty projects out of the 94 planning grants were rated on the 
extent of implementation achieved by the project. The appraisal team 
judged claims for success in implementation were "overstated" by one 
project, "ordinary" for ten others, and "trite or insignificant" in four 
projects. In the case of 24 other projects, implementation seemed to 
flow logically from the program, and in 21 projects implementation of 
accomplishments was rated as significant. 

Eighteen of the operational projects indicated that plans for im- 
plementation had been built into their projects. After evaluation, the 
review team judged that six out of the 18 projects had plans for imple- 
mentation that likely were successful. Three of the project^' imple- 
mentation plans were judged to flow logically from the program itself; 
however, eight other plans were judged as being "ordinary," while one 
was considered "trite or insignificant. " Fifty percent of the plans for 
implementation given by the projects were judged a.s ca.pable of suc- 
cess, and another 50 percent were of such poor quality that failure 
seemed assured. 



Provisions for Continuation 
For planning projects 

Fifty- nine out of the 94 grantees indicated they have given some 
consideration to grant continuation and support by means other than 
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ESEA Title III funds, and 35 projects indicated that no consideration 
had been given for grant continuation. 

Of those wanting to continue, 30 projects appeared likely to 
succeed in this respect. Fifteen other projects planning or wanting to 
continue were judged as unlikely to succeed. Five projects submitted 
quite sketchy plans for continuation, and in six instances, the plans 
were too nebulous to be appraised by the analysts. Three projects 
failed to respond in this category. 

F or operational projects 

A rather large number — 33 out of the 43 — indicated that speci- 
fic plans had been, or were being made, for project continuation after 
termination of funds. Seven projects indicated that they had formulated 
no plans for continuation, and three projects made no response. 

In rating continuation likelihood, 12 of the 33 projects that had 
plans seemed likely to make it. Eleven presented continuation plans 
that were judged as questionable, five had plans that were at best 
“'quite sketchy, " and ten projects had plans that were too nebulous to 
appraise or evaluate. From this analysis, probably 12 to 15 projects, 
at best, will continue, through local efforts and funding, either in part 
or in toto. 



The following table summarizes the provisions for continuation: 
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Question 


Planning 


Operational 


Total 


Does the project give at 
least some consideration 
to continuation by means 
other than support by ESEA 
Title III? Yes 


59 


33 


92 


No 


35 


7 


42 


No response 






3 


Totals 


Ji 


43 


137 



If yes, are these considera- 
tions (plans) 

a. well planned and likely to 
succeed 


30 


12 


42 


b. questionable 


15 


6 


21 


c. quite sketchy 


5 


5 


10 


d. too nebulous to appraise 


6 


10 


10 


e. no response 


_3 


_g 


3 


Totals 


59 


33 


92 



EVALUATION PROCEDURES 



The first guidelines did not give evaluation the attention that 
was the case in later editions, yet some mention was made of it and 
some expectations for evaluation were evident. 

This section covers these items: 

Types of evaluative procedures used 
Relationship between evaluation and overall project 
Cost of evaluation 

Adequacy of evaluation procedures 

Types of Evaluative Procedures U s e d 
For planning g rants 

V 

Thirty-two projects stated that surveys (questionnaires) were 
the key evaluative methods used. Only four projects indicated that re- 
search design procedures figured in the evaluative techniques, and 
four other projects used pre- and post-tests. Fifteen grantees listed 
interviews as the main evaluative procedure, one listed opinions, four 
listed outside consultants, one conferences, two listed time studies, 
and one statistical analysis. But 45 projects gave no evidence or failed 
to respond to this question. 

For operational grants 

The survey or questionnaire technique reportedly was used by 
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17 projects, and 16 projects used pre- and post-test evaluative proce- 
dures. Nine used interviews, two opinion surveys or polls, and four 
used as the methodology and instrument for evaluation the official 



terminal reporting form required of ending Title III Grants. 



The following table summarizes the findings on types of evalua- 



tive procedures used: 





❖ / 

Type of procedure" 


Planning 


Operational 


a. 


survey (questionnaire) 


32 


17 


b. 


interview 


15 


9 


c. 


research design 


4 


4 


d. 


pre-test, post-test 


4 


16 


e. 


outside evaluators 


4 


0 


f. 


time studies 


2 


0 


fe- 


opinion 


1 


2 


h. 


conferences 


1 


0 


i. 


statistical analysis 


1 


0 


j- 


no response or notindxcated 45 


__0 




Totals 


109 


48 



Total 



49 

24 

8 

20 

4 

2 

3 

1 

1 

45 

157 



Relationship Between Ev? luation and Overall Project 
For planning grants 

Eighteen projects were judged as having evaluation integrally 
related to the overall project; and 20 had evaluation as only a part, or 
slightly related, to the overall project. Thirty- seven projects had no 
particular relationship between the evaluation and the overall project. 



— Some, projects used more than one evaluative procedure 
ther eby accounting for the numbers in the above table exceeding the 
total number of projects in the sample. 



o 



and •. one case evaluation was forced to meet the overall characteris- 
tics and nature of the project. Eighteen projects failed to respond or 
indicate any relationship between evaluation and the overall project. 

For operational grants 

Twelve of the 43 operational projects included evaluation as an 
integral part of their projects, and 14 projects had some provisions 
for evaluation as a part of their overall project planning. Another 14, 
however, were appraised as having no evident relationship between 
evaluative procedures used and the overall project. In one project, 
the evaluation had been forced to correspond to the project's character- 
istics, and two projects made no response in this area or had such 
poor provisions for evaluation that they were incapable of being an- 
alyzed. 



The following table summarizes the relationships between 



evaluation and the overall project: 
Type of relationship 

a. Not any particular rela- 
tionship, or very casual 

b. A part 

c An integral part 

d. Evaluation appears forced 
to need the guidelines re- 
quirement 

e. No response 

T otals 



Planning 


O per ational 


Total 


37 


14 


51 


20 


14 


34 


18 


12 


30 



1 


1 


2 


18 


__2 


20 


94 


43 


137 
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Co st of Evaluation 

So few of the 137 planning and operational projects listed ex- 
penditures for evaluation that any statistical summary is meaningless. 
This omission emphasizes once again the lack of concern and under- 
standing of the importance of evaluation. The failure to formulate 
plans for evaluation is a major characteristic of a large majority of 
the reports, and this matter is considered in the recommendations. 

Adequacy of Evaluation Procedures 
For planning proj ects 

Only 30 of the 94 projects .vere judged as having adequate 
evaluation procedures in the project design. This may be contrasted 
to 31 projects which gave no evidence whatsoever of evaluation proce- 
dures. Eheven projects indicated some provisions for evaluation, and 
eight gave a little evidence of evaluative considerations. Fourteen pro- 
jects failed to respond to this category. Ol <4 ijrojects, 49 gave 

indications of planning for evaluaton I' tngi from little to much; 45 
gave no indication of evaluation, or they failed to respond in this area. 
For operational proiects 

Only five of the 43 operational projects presented adequate 
evaluative procedures in their project designs. Another 20 projects 

f 

presented some evidence that efforts or provisions loi* evaluation had 
been considered. Ten projects were judged as giving scant attention 
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or paying "lip service" to evaluation, and Six others gave jio evid 3nce 
of provisions for evaluation. Two projects made no response in this 
area. 

A little over eight percent of the projects had made plans and 
provisions that promised to be adequate for evaluation of their pro- 
jects; about 70 percent (or 30) had done a little, and about 13 percent 
had not bothered with evaluation at all. 

The following table summarizes the adequacy of evaluative pro- 
cedures: 



Degree of adequacy Planning Operational T otal 



a. much 


30 


5 


35 


b some 


11 


20 


31 


c. little 


8 


10 


18 


d. no evidence 


31 


6 


37 


e. no response or not indicated 


ii 


_2 


16 


Totals 


94 


43 


137 
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MAJOR PROBLEMS 



The study team judged each project in terms of this question: 
’jB'rom your study of the report, what were two or three major prob- 
lems that seemed to be implicit or explicit? 

Fo r planning projects 

Some 20 individual problems were identified, and two major 
problems were: (1) financing the project, mentioned 12 times, with 

the primary concern being that of finding additional funds for project 
continuation; and (2) problems associated with public relations, 
listed 10 times, which included matters such as bringing about public 
a , rareness of the projects and their intended accomplishments. 

Five other problems were identified, although mentioned less 
requently but were still considered important. These were: (1) fail- 

•’ c to formulate clear objectives, listed eight times; (2) lack of plan- 
dug, listed seven times; (3) lack of qualified personnel, also listed 
seven times; (4) problems related to implementation, mentioned six 
times; and (5) lack of official cooperation from the schools, also 
oientioned six times. 

Of the 13 other problems that were gleaned from the 94 reports, 
only one occurred as many as three times, while four were mentioned 
t,a o times, and eight w'ere listed only one time eacn. 
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Twenty-nine projects, or about 30 percent, were judged as 
i. ivirig no major or minor problems, and 11 projects were so vague 
,*r r<mtained so little information that no problems could be identified, 
i' t ) r operational projects 

Again, the problem of grant continuation led all others in ire- 
qruau-y, with 13 projects giving it a major status. In all, 15 problems 
er(‘ identified. Next was the problem of maintaining and increasing 
-.fjident interest, which was mentioned five times. The following prob- 
iorus were listed four times each: (1) public relations or selling the 

p\iblic and schools on the project, (2) the problem of overextension 
or ;iUempting to do too much with their limited resources, and (3) the 
l>rMhlem of overspending. 

Several projects felt that they were in danger of exhausting 
!■( budgets. Lack of adequate planning was listed as a major prob- 

0 HI in three instances. Four other problems were listed two times 
. .i- h. and five additional problems each were listed once. 

nly four projects were judged as having no major problems, 
other projects did not contain sufficient information to ascer- 

1 .i!? it they did or did not have problems. 

The following table summarizes the major problems found in 

“ ‘-t; i 37 projects: 



o 
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Problem Planning 


Operational 


Total 


a. Grant continuation 


12 


13 


25 


b. Public relations 

c. Failure to formulate 


10 


4 


14 


clear objectives 


8 


0 


8 


d. Lack of planning 


7 


3 


10 


e. Lack of qualified personnel 


7 


o 

Ut 


9 


f. Implementation 


6 


2 


8 


g. Evaluation 


3 


0 


3 


h. Poor facilities 


2 


0 


2 


i. Student interest 


2 


5 


7 


j. Overextension of effort 


1 


4 


5 


k. Dissemination 


1 


0 


1 


1. District cooperation 


1 


0 


1 


m. Inadequate leaderf-hip 


1 


2 


3 


n. Overspending 


_0 




4 


Totals 


61 


39 


100 
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OVERALL APPRAISAL 



This overall appraisal by the study team was based upon 
several approaches. The appraisal included a composite summary 
of the checklist instrument as well as a complete introspective analy- 
sis of each project. Included as criteria to be rated were such items 
as completeness planning, identification of o’' jectx'.’-f and methodology, 
involvement of local persons as v/ell as outside consultants, provisions 
for continuation, and probable overall success of the project. 

For planning projects 

The study team made an overall appraisal of the project, as 
gleaned from the report. Fourteen were rated as outstanding, 37 as 
good, 19 average, 18 poor, and 6 very poor. Thus, 51 (or 54.2%) 
of the 94 were rated either outstanding or good. Nineteen (or 21%) of 
the projects ranked average, and 24 or 25 percent v/ere judged as poor 
or very poor. 

For operational projects 

In final appraisal of the 43 projects, five projects were rated 
as outstanding, and 20 were judged as good. Thus, a total of 25 pro- 
jects were considered outstanding or good. (Since all of the above 
grants were terminated, the evaluators did not include such items as 
failure to plan for continuation or provisions for adequate evaluations 
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as increments in their evaluation. ) The major concerns were identify- 
ing program objectives and how well these were accomplished. Of 
coursej the apparent contributions and changes in the schools and the 
students as a result of the Title III project were also important consid- 
erations . 

The following table summarizes the final appraisal of the 137 
projects: 



Rating 


Planning 


Operational 


Total 


a. outstanding 


14 


5 


19 


b. good 


37 


20 


57 


c. average 


19 


11 


30 


d. poor 


18 


7 


25 


e, very poor 


6 


_0 


6 


Totals 


94 


43 


137 



o 
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R ECOMMENDATIONS 



Based upon this study, the following recommendations for fu- 
ture action are suggested: 

I. EVERY PROJECT PROPOSAL SHOULD BE REQUIRED TO 
SU BMIT EVIDENCE THAT THOSE DEVELO PING THE PRO- 
JECT HAVE A GOOD GRASP OF THE LOCAL AREA , BOTH 
IN TERMS OF NEEDS AND RESOURCES . 

This evidence does not have to be a needs assessment 
study if the project is a single idea or a program; still, some evi- 
dence of local considerations should be evident. 

II. EVERY PROPOSED SUPPLEMENTARY CENTER TYPE OF 
PACE PROGRAM SHOULD INCLUDE A REASONABLY 
THOROUGH NEEDS ASSESSMENT STUDY . 

A statement or assessment of needs should include the 
identification of goals, processes for goal attainment, and specifica- 
tion of areas of greatest needs and deficiencies. Most terminal re- 
ports included little or nothing about how issues or programs were 
selected, and, in many other instances, this essential aspect of the re- 
port covering the supplementary center type of project was either 
minimized or omitted. 
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ll!. STATE DEPARTMENTS OF EDUCATION NEED TO GIVE, 

CAREFUL CONSIDERATION TO THE TYPE OF TERMINAL 
REPORTS THAT WILL PROVIDE A FITTING CLIMAX TO A 
P ACE PROJECT . WILL MEET LEGAL REQUIREMENTS OF 
REPORTING, AND WILL ALLOW ESSENTIAL FINDING S TO 
BE DISSEMINATED EFFECTIVELY . 

Our study found that most projects omitted one or more 
types of information, such as: project title, type of prcject, grant 

number, period of time, amount of the grant, number of students to be 
served, cost per student, number of school districts in/olved, the 
name of the state, and so forth. 

The study team had no idea how sloppy, inaccurate, and in- 
complete it would find the final reports, in most cases . hose who 
submit such reports are guilty of professional negligence and fiscal ir- 
responsibility, and they need to be dealt with accordingly. If this 
message seems overstated, one needs only to examine the end of pro- 
ject reports submitted by many projects. 

IV . INVOLVEMENT OF COMMUNITY RESOURCES AND PERSON- 
NEL SHOULD BE MORE CAREFULLY CONSIDERED ; IT 
SHOULD BE REALISTIC AND SHOULD HAVE ADEQUATE 
FOLLOW - THROUGH . 

No promises should be made that cannot be kept; no obli- 
gations should be incurred that cannot be met. 



I 
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A majority of the projects studied are guilty of overexten- 
sion and superficiality on community relations. The project developers 
promise too much, involve too many, and analyze too little the HOW of 
effective community involvement. 

V. ALL PROJECTS SHOULD HAVE EFFECTIVE EVALUATION 
PROCEDURES — EFFECTIVE IN TERMS OF STATED OBJEC- 
TIVES AND PLANNED PROGRAMS. 

The call for better evaluation is an old saw, if the three 
years of PACE history is old, but the call needs to be made again and 
again. 

In only one or two instances out of 94 planning projects can 
one glean from the proposal a serious and sophisticated concern about 
evaluation — a concern that viewed evaluation as a vital part of the day- 
to-day monitoring process as well as a judgmental decision reflecting 
success or failure of the program. 

VI. EVERY PACE PROPOSAL SHOULD HAVE A SEPARATE BUD- 
GET ITEM FOR EVALUATION , AND THIS FIGURE SHOULD 
NOT BE LESS THAN FIVE PERCENT OF THE TOTAL BUD- 
GET . 

Only a small number of the terminated projects included 
plans for evaluation, and even these appeared to be afterthoughts or 
were non-integral parts of the project structure. While there is rea- 
son to believe that evaluation has improved during the last year, the 
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level is still far below what is desirable and what is needed. 

VII. MORE EVIDENCE OF PLANNING SHOULD BE REQUIRED IN 
FUTURE PACE PROPOSALS . 

The study team was aware of the dilemma between over- 
planning and over - structuring on the one hand, and a relaxed, prag- 
matic approach to design on the othfer, but evidence gained from the 
137 terminal reports lead ^ls to believe that greater emphasis should 
be placed on planning and design in all future proposals. 

VIII. PROVISIONS FOR CONTINUATION AFTER TERMINATION OF 
ESEA TITLE III FUNDING SHOULD BECOME MORE EVIDENT 
IN THE FUTURL . 

The newness of PACE, the unexplored parameters of its 
guidelines, and the unknown labyrinths of federal assistance have all 
mitigated against serious consideration of what might take place when 
the planning grant ended. But as we look ahead, profiting from the 
past, continuation considerations should become more important with- 
out becoming a r equijrement for approval. 

IX. FUTURE PACE PLANNING GRANTS SHOULD BE ALLOCATED 
ON A SHARING BASIS WITH LOCAL COMMUNITIES -— SOME- 
THING IN THE DOLLAR RANGE OF 8 OR 10 TO ONE . 

The study team found that where local funds were com- 
mitted to the project, it was better planned, the objectives more 
clearly stated, and the procedures for realizing the major goals of the 
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project more adequately outlined. Furthermore, the study team be- 
lieved that a local share of the project expenses — even if small en 
hanced prospects of local continuation after termination of federal 
money. Also, dissemination and implementation are expedited by a 
sharing basis, and a more receptive climate is created for continua- 
tion. 




APPENDIX A 



APPENDIX A 



EVALUATION INSTRUMENT 
Analysis of 

Terminated ESEA Title III Operational Programs 

A. Project Characteristics 

1. Grant number 

2. Period of project 

3. Total amount of grant 

4. Total Non-Federal support 

5. Total Federal support under Title III 

6. Total Federal support other than Title III 

7. Major description of project (c’ .eck one only) 
a. Innovative 

b. Exemplary 

c. Adaptive 

8. Types of activity (check one or more) 

a. Planning of program 

b. Planning of construction 

c. Conducting pilot activities 
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d. Operation of progra,m 

e. Constructing 

f. Remodeling 

Project title (5 words or less) 



State of applicant^ 



Scope of project (check one only) 



a. Nati/bnal 



b. Regional (multi- state) 



c. Ohe state 



d. One district 



e. Multi-district; meaning two to six districts 



f. Multi-district (or county); meaning seven or 
more districts within one state 



g. A few schools within one district 



h. One school 



Area served by project (best fit one ) 



a. Urban 



(1) central city 

(2) urban fringe 

b. Other urban (suburban) 

(1) cities of 10, 000 or more 

(2) cities of 2, 500 to 10, 000 
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c. Rural 

(1) places of 1, 000 to 2, 500 

(2) other rural 

Type of activity (check only one) 

a. Mostly services to several local school districts, 

such as administration, dissemination, planning, 
and developing. 

_____ b. Mostly instruction or services to pupils, such as 
arts, sciences, social studies, mobile demon- 
strations, museum visits, ETV, outdoor camp- 
ing, and guidance and counseling. 

c. Mostly services to teachers and principals — 

services such as instructional materials, 
audio-visual, demonstrations, and in-service 
training. 

d. Mostly planning, such as surveying needs, de- 
signing new programs, and visiting innovations. 

e. Mostly installation of one or two innovations in 

one or two schools — innovations such as ITA, 
computer- as sisted instruction, teaching aides, 
parent involvement, job placement, and new 
courses . 

Project classification 

a. Focusing upon a single idea or program. 

b. Serving as a supplementary education center 

with several activities. 

School enrollment, etc. (Section C in guidelines) 
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S)CHObL liMROLLMENT AND PROTECT PARTI C I PATtON DATA 



TkTAFT 

MeKBEAS 

EUAH6C0 

lU 

lU'ieitficc 

TRAIMIllfc 

Foe 

ncoiecr 



1. dCUbBL 

ENR0U.llC«r 

IN seM«KA7Hic 
Ml>iA SCKVtb 



(ttl 



Z. 

persons 

TERMED 

PROietT 



«a-) 

niM-te 



3. 

NAb triONAL 
PEebONS 
NEEDlUb 
SCR'IICt 



H 



PUftUC 



Tb? 

NAM-PUALtC 



'o' 

NtM-rubUc 



MOT 

CNMLLEb 



ARe- 

KiMbae. 






PUNl.lt 



cm 

MON-ri)kLlC 



ennollcd 



kimder. 



Total. Nunaeit op 
PAK.TIC.IPAMTS BV TVPtt 
(AfPLlCKBuC TO PICiURCC 
atVCiJ IM C-Z A0OMC) 



1'6 



7-11 



ADULT 



OTHER. 



TOTALt 



VMHlTe 


NC^RD 


AHrRICPN 

tMDIKH 


CVUEK 
NOM -'MHlTE 


TCTMU 










1 
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16. Number of persons served and estimated cost distribution 
(Section E in guidelines) 



B. Project Accomplishments (What do the authors contend was 

achieved? What appraisal of this 
is made by reviewer? , 



], . Research 

Accomplishment ‘ Appraisal 

1 . 

2 . 

3. 

4. . , 

2. Product development (such as new program for social 

studies, new materials for handi- 
capped) 

Accomplishment Appraisal 

1 . 

2 . ' 

3 - 

. 4. 
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NUMBER or PERSOMS SERVED OR TO BE SERVEO ANO ESTIMRTtD COiT DISTRIBUTION 





TDTKL NUMBER SERVED OR TO BE BEl 


ftVED 


NON- PUDLY 
SCHOOL 
PUPILS 
TNCLUflCD 

- SD 


ESTtHKieD 

HOST 






PRE-K 

a> 


H 


1-k 

LiA^ _ 


*T*IZ 


ADULT 
/SI ■ 


OTHBR 

c<*i 


CVltkl-UATtUC PR0&k^S 
UlObriCIRMCY SUNVCV (ARCK NCiD»> 


















M cuniiieuLUH RcouiRCncNn stury 

UNCUIMNS RiANMIMb r»R FUTttRCNKCI4 














V A 




(c) RKboURCC RVAIURmiLITY 
AND UTILItATlDM RTUblCD 


















XMSTRUCTION AND/oK eNRtCMMBuT 


















(e'l FOReiCM LANSUABBt 


















(%) LANGUAGE ARTGCENGUtN inARtVtHCW 


















NEMCbtAL RCAbING 


















(h) MATHSHAYiea 


















1*0 SCIENCE. 


















(j) SOCIAL stuoies/hum AN tries 


















(W PHYSICAL PlTNeSS/ReCREATlOK 


















V 

VOCATIONAL AHDUSTIttAL ARTS 


















1 

1 

(Vh^ SPECIAL- YHYSICALLY UbMOICATRED 


















(y>) SPECIAL -mentally RETARbeD 


















(o) SPECIAL •IHSTURDebilHCL »EUNW»*ri 


















Cp\ DRorour 


















SPECIAL- MINORITY CROUPS 


















THSTRUCTION ADDEWDA 
(ir) ebuciktionau T\i/ltAbio 


















(S) AUtlO-VISUAL AIDS 


















(Cl DEHONSTRATIOH/lCARNINC CCNTRRS 


















(u^ LIERKRY FACILITie>. 


















(v> HATKIUAL AHO/ofC SCKVtCft CeWTCWt 


















l¥4) DIKTIS rftOC«S«lMfi 


















PERSONAL SERVICES 

(R'l MCblCAL^ T>e**TAL 


















lip SOCIAL/fSYCUOLOCiC AL 


















OTHER 
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3. Skill development (such as reading, mathematics, voca- 
tional) 

Accomplishment Apprais^ 

1 . 

2 . 



3. 



4. 

4. Process development (such as inquiry training) 

Accomplishment Appraisal 

1 . 

2 . 

3. 

4. 

5, Program development (such as nongraded school, team 

teaching) 

Accomplishment Appraisal 

1 . 

2 . 

3. 

4. 
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6. Pupil relations (such as guidance and counseling) 

Accomplishment Appraisal 

1 . 

2 . 

3. 

4. 

7. Community relations (as fostered by project) 

Accomplishment Appraisal 

1 . 

2 . 

3. 

4. 

8. In - service programs for teachers and / or administrators 

Accomplishment Appraisal 

1 . 

2 . 

3. 

4. 

9. Demonstration (of a program, such as team teaching) 

Accomplishment Appraisal 

1 . 

2 . 

3 . 



o 
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10. Dissemination (spreading "the word" to others) 

Accomplishment Appraisal 

1 . 

2 . 

3. 

4. 

1 1 . Implementation 

Accomplishnaent Appraisal 

1 . 

2 . 

3. 

4. 

C . Provisions for Continaa.tion 

1. (yes or no) Does the project give at least some 

consideration to continuation by means 
other than support by ESEA Title III. 

2. If yes, are these considerations-- 



a. 


well planned and likely to 
succeed 


b. 


questionable 


c . 


quite sketchy 


d. 


too nebulous to appraise 
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D. Design Project as Evidenced from Report 

1. Degree to which evaluation procedures are related 

to the objectives 

a. (much), b. (some), c, (little), d. (no evi- 
dence of relationship) 

2, What evaluative procedures are used? (research 
design; pre-test, post-test, survey, interview, 
etc. ) 



3. Estimated cost of evaluation 

4, What relationship does evaluation appear to have in 
terms of the overall project? 



a. 


An integral part 


b. 


A part 


1 ^ • 


Not any particular relationship, or very 
casual 


d. 


Evaluation appears forced to need the 
guidelines requirement 



E. Final Appraisal 

1. What one or two ideas, programs, etc., do you consider 
significant and worthy of widespread dissemination? 



2. From your study of the report, what were two or three 
major problems that seemed to be implicit or explicit? 
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3. What is your overall appraisal of the project as gleaned 
from the report? 

a. outstanding 

b. good 

c. average 

d. poor 

e. 



very poor 



Addendum for Terminated Planning Grants 



ment, 

1 . 



Planning projects will focus on (1) an extensive needs assess- 
or (2) upon the development of a particular idea or program. 



Needs assessment 



a. (yes or no) Did the planning grant include a needs 
assessment study? 

b. If answer is "yes, " then respond to the following 
items: 

1, Which of these data were used in the assess- 
ment? (check appropriate ones) 

(a) Community factors 

(i) Rural-urban composition 

(ii) Median family income 

(iii) Median grade attained 

(iv) Socio-economic class 
structure 

(v) Employment patterns 

(vi) Horizontal mobility 

(vii) Per pupil expenditure 

(viii) Attitudinal considerations 

(ix) Other 



(b) School factors 

(i) Student achievement mea- 
sures 

(ii) Socio-economic class 

structure of school sys- 
tem 

(iii) Composition of student 

body 

(iv) Other^ . 
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Z. What procedures were employed to determine 
neede.? (check appropriate ones and write in 
other s) 

(a) Questionnaire 

^b) Interview 

(c) Discussion groups 

(d) Study groups 

(e) Research design 

(f) Other 



3. Who determined the needs? 



(a) 


Members of present staff 


(b) 


New personnel 


(«) 


Special study team 


(d) 


"Outside" consultants 


(e) 


Other 


Is systematic planning in evidence? 


(a) 


Yes 


(b) 


No 


(c) 


Yes and no 


(d) 


Cannot tell 



5. Are any planning procedures used, such as; 

(a) system analysis 

(b) PERT " 

(c) Critical Path Method 

(d) sequences or phases 

6. Are evaluation procedures built into i;he plan- 
ning procedures? yes no 

2. Development of a particular idea or program 

a. Can you perceive what the project has in mind? 

(1) yes, clearly 

(2) yes 

(3) somewhat 

(4) little clarity 

(5) for all practical purposes, I cannot 
perceive this from the report. 
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b. Has the report presented the idea or program in a 
concise and understandable manner? 

(1) yes, clearly 

(2) yes 

(3) somewhat 

(4) little clarity 

(5) for all practical purposes, the presen- 
tation shall be classified as imprecise 
and confusing. 

The purpose of a planning grant is to develop a proposal that 
can be turned into an operational grant. In about 75 percent of the 
cases this has happened. 

3 . Based upon this report , what kind of _a picture do you have 

about the next stage — the operational grant ? 

(a) clear picture 

(b) some clarity 

(c) uncertain 

(d) very little clarity about operational 

phase 

(e) evidently no operational grant is 



planned 




